Remote color matching process and system

ABSTRACT

A method and system for cost effective, convenient remote color reproduction and matching that can be used to convey color to observers remote to the physical source of color. Such remote observers can include product consumers wishing to view a product color, for example. In a preferred embodiment, the method comprises capture of article or product reflectance spectra and the use of this spectrum to filter ambient light or directed light in the environment of a remote user. Other embodiments of methods include various techniques to capture product spectral information and color matching functions useful for color reproduction using colored light sources. Additional systems embodiments include devices exploiting multiprimary displays to render the product color in avoidance of metamerism.

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application claims the benefit of U.S. Provisional Ser. No.62/708,073 filed Dec. 1, 2017. This application is acontinuation-in-part of U.S. patent application Ser. No. 16/350,553filed Nov. 30, 2018. Both of the above applications are incorporated byreference herein.

BACKGROUND

There are numerous applications for remote high fidelity reproduction ofcolor. In order to achieve such reproduction, attention must be given toillumination spectra, the variation of the human visual response tocolor as well as metameric, and other effects that occur in humanvision.

Color reproduction is a daunting challenge in the remote display ofproduct colors. Often, online and print (catalog) depictions of productcolors are insufficient to render the same color as perceived by theconsumer upon direct inspection of the given product. According toInvesp Infographic regarding online return rates statistics, at least30% of all products ordered online are returned compared to only 8.89%bought in brick-and-mortar shops(https://www.business2community.com/infographics/e-commerce-product-return-statistics-trends-infographic-01505394).Some significant contribution to these returns in the case of appareland shoes are product colors that are deemed unsatisfactory by theconsumer.

Currently, domestic shopping through online retail stores comprises onlyabout 10 percent of consumer activity, but this percentage is slated toincrease in the coming years. For consumers making online purchases, oneof the key product features that could benefit from improvedspecification is color. This capability would extend to product sales onEbay or Etsy wherein the general public could determine an item's colorfor posting with the item for sale.

In addition to the problem of accurate reproduction of product colorsfor the consumer, it remains challenging for the consumer to match thecolor of an advertised product to the color of an item in the consumer'spossession. Many variables contribute to the difficulty in high fidelityreproduction of color as well as in color matching. Among these are:

-   -   Variable illumination    -   Observation angle    -   Observer metamerism    -   Spectral matching versus colorimetric matching—lack of spectral        resolution    -   Background    -   Textures    -   Temperatures

Hence, it would be advantageous for a method and associated system thatwould overcome these most critical of these issues and present a highfidelity remote representation of the true article or product color.Foundational to development of such technology is an assessment of thedetails concerning how color is defined and perceived by humans.

Quantifying Color

The classical descriptive mechanism for accurately defining therelationship between the wavelength(s) of a color and the perceivedeffect on the human eye is a color space. This construct also permitscolor comparisons between displays that exhibit different ways theydisplay color, i.e. color profiles.

Color Spaces

Color descriptions are predicated on either additive or subtractivecolor theory; the former for transmitted light (ex. electronic displays)the latter for reflected light (printed materials and paints).

A color space describes an abstract, multidimensional environment inwhich any particular color can be defined. A color model is a geometricor mathematical framework that attempts to describe the colors humansperceive. It uses numerical values pinned to dimensions of the model torepresent the visible spectrum of color. A color model provides a methodfor describing, classifying, comparing, and ordering colors.

Further, a color space is a practical adaptation of a color model thatspecifies a gamut of colors that can be produced using that model. Thecolor model determines the relationship between values, and the colorspace defines the absolute meaning of those values as colors. Thesevalues, called components, are in most instances floating-point valuesbetween 0.0 and 1.0 (Introduction to Color Programming Topics for Cocoa,Apple Programming Guide).

There are five major color models or spaces that sub-divide into others,these are: CIE, RGB, YUV, HSL/HSV, and CMYK; the latter being asubtractive color model applicable to printing. These standardized colordescription systems are used to quantify color and permit consistentproduction of print, paint, and video display colors. Transformationamong these different color spaces is achievable mathematically (AdrianFord and Alan Roberts, Color Space Conversions, Aug. 11, 1998,http://www.wmin.ac.uk/ITRG/docs/coloreq.html, page 1-31.).

Reference is made to FIG. 1 , which depicts the 1931 Commissioninternationale de l'éclairage (CIE) Chromaticity diagram as arepresentation of a color space encompassing all humanly discernablecolors.

The XYZ chromaticity diagram is a standard color space, independent ofany choice of primaries, in which the color of any object or light canbe specified, independent of its total reflectance or brightness. Thehorseshoe-shaped perimeter of this space corresponds to all saturatedcolors, i.e. single wavelength (pure) colors. It can be said that as atrajectory is traced inward from these perimeter coordinates, theeffective optical bandwidth of the light represented by the coordinatesincreases and becomes unsaturated (impure). In accordance withGrassman's Laws of Additive Color Mixture, primary colors selected onthis diagram establish a polygon wherein any color enclosed in therespective polygon can be generated with the appropriate intensityweighted combination of the primary colors. The space of colors that canbe synthesized by a set of primary colors is called the gamut.

FIG. 2 depicts the color gamut enclosed by the dotted triangle 1, usingred 5, green 3, and blue 7 primary colors inherent is various displayand reflective systems. FIG. 3 depicts expansion of the color gamut 11through use of a greater plurality of primaries, 13, 15, 17, 19, and 21.Again, it should be emphasized that primaries whose coordinates arelocated on the perimeter of the chromaticity diagram are singlewavelength (fully saturated). Such saturated primaries could beapproximated by laser sources with very narrow bandwidths. As thecoordinates of a primary approach the center of the diagram, the primaryis of increasing optical bandwidth (unsaturated). The very center of thediagram represents the most unsaturated color—white, encompassing allvisible wavelengths.

Visual Response

In the XYZ color space, the Y coordinate represents luminance (measuredintensity). It is useful to transform this space to an Yxy color spacein which the Y coordinate remains representative of luminance, and wherex represents hue, and y represents saturation.

XYZ tristimulus values and the associated Yxy color space form thefoundation of present CIE color spaces which are widely used for colorcomparison. The concept for the XYZ tristimulus values is based on thethree-component theory of color vision, which states that the eyepossesses receptors for three primary colors (red, green, and blue) andthat all colors are seen as mixtures of these three primary colors. TheXYZ tristimulus values are calculated using these CIE Standard Observercolor matching functions (CMFs) x(λ), y(λ), z(λ), as depicted in theFIG. 4 .

These functions represent the spectral response (across the visiblespectrum from 380 nm to 780 nm) of the three types of conephotoreceptors in the eye and have been generated as an ensemble averageacross a population of individuals.

The Commission International de l'Eclairage (CIE) has documented CMFsfor two different categories of standard observers: a 2 degree 1931 CIEstandard observer and a 10 degree 1964 CIE standard observer. Thesematching functions are ensemble averages across a population of normalobservers using viewing conditions that vary emphasis on the fovealresponse. In FIG. 5 , a plot of RGB CMFs, the variation of response fromobserver to observer is illustrated by the spread in response across aparticular ensemble of 49 observers. (Stiles, W. S., & Burch, J. M.,“NPL color-matching investigation: Final report”, Optica Acta, 6, 1-26,1959.)

The Yxy encoding is a very good color space solution due to its strongphysical/perceptual background. One can go from RGB color space to XYZ(selecting a certain color-space transform matrix), and then go from XYZto Yxy using the following formulas:x=X/(X+Y+Z)y=Y/(X+Y+Z)

${X = {K{\int_{380}^{780}{{S(\lambda)}{\overset{\_}{x}\,(\lambda)}{R(\lambda)}d\lambda}}}}{Y = {K{\int_{380}^{780}{{S(\lambda)}{\overset{\_}{y}\,(\lambda)}{R(\lambda)}d\lambda}}}}{Z = {K{\int_{380}^{780}{{S(\lambda)}{\overset{\_}{z}\,(\lambda)}{R(\lambda)}d\lambda}}}}{K = \frac{100}{\int_{380}^{780}{{S(\lambda)}{\overset{\_}{y}\,(\lambda)}{R(\lambda)}d\lambda}}}$Where S(λ): Relative spectral power distribution of the illuminator

-   -   x(λ), y(λ), z(λ): Color-matching functions for CIE 2° Standard        Observer (1931)    -   R(λ): Spectral reflectance of specimen

The CIE chromaticity diagram is generated by plotting the average CMFsin the x,y coordinates. In the Yxy color space, Y remains the luminanceand independent of luminance, the x and y coordinates represent hue andsaturation respectively. Other color spaces have been defined that arelinear transformations of the CIE 1931 color space. For quantifyingcolor differences, a more uniform color space with u′, v′ coordinateswas derived; in the associated coordinates Δu′v′≤0.002 is assessed aschange that is visually undiscernible to humans.

While the 1931 x, y chromaticity diagram is accepted and used widely inthe field of color science, there are a few fundamental flaws. One ofthe major problems is the non-uniformity of the diagram. A certaingeometric distance in, for example, the green part of the diagram doesnot represent an equal perceived difference in color as the samedistance does in the blue part of the diagram. In 1942, MacAdam(MacAdam, D. L., “Visual Sensitivities to Color Differences inDaylight”, Journal of the Optical Society of America, 32(5), 247, 1942.)did a series of color matching experiments to determine the justnoticeable differences (JND) of chromaticity. MacAdam shows theresulting JND plotted in the x, y color space are in fact ellipses withwidely varying size depending on their location in the chromaticitydiagram. In FIG. 6 , these ellipses are plotted in the x,y color space,but enlarged 10 times for ease of viewing. It is apparent thatperceivable color differences and geometrical distances between colorcoordinates depend on the location in the diagram itself (W. Hertog,“The design and implementation of a spectrally tunable LED-based lightsource: towards a new era of intelligent illumination”, PhD Thesis,Department of Optics and Optometry of the Universitat Politècnica deCatalunya, December 2016).

To achieve a sense of human visual sensitivity to wavelength changesacross the visible spectrum, reference is made to FIG. 7 . This is aplot of JNDs in color across the visible spectrum for saturated light(Krudy A, Ladunga K, “Measuring wavelength discrimination thresholdalong the entire visible spectrum”, Period Polytech Mech Eng 45, 2001,pp. 41-48.). It is apparent that variation in wavelength between lightsources as small as 2 nm can be detected.

In quantizing the intensity of lighting, gamma encoding of images isused to optimize the usage of bits when encoding an image, or bandwidthused to transport an image, by taking advantage of the non-linear mannerin which humans perceive light and color. The human perception ofbrightness, under common illumination conditions (not extremes), followsan approximate exponential power function with greater sensitivity torelative differences between darker tones than between lighter ones,consistent with the Stevens' power law for brightness perception. Ifimages are not gamma-encoded, they allocate too many bits or too muchbandwidth to highlights that humans cannot differentiate, and too fewbits or too little bandwidth to shadow values that humans are sensitiveto and would require more bits/bandwidth to maintain the same visualquality. Gamma encoding of floating-point images is not required (andmay be counterproductive), because the floating-point format alreadyprovides a piecewise linear approximation of a logarithmic curve. In thepresent application that involves single pixel display for colormatching, gamma encoding is not required.

It is important to recognize that color perception is a psycho-visualphenomenon, so certain viewing conditions must be under control toachieve consistent color reproduction at the stage of human perception.

Metamerism

Two or more stimuli having identical chromaticity coordinates, but adifferent spectrum, are called metamers. The stimuli can be either lightsources or objects reflecting or transmitting a certain illuminationspectrum. Metamerism exists because the retinal cones are tristimulusreceptors, which means that for one set of chromaticity coordinatesthere are an infinite number of matching spectra. Metameric failureoccurs when a change of the illuminant spectrum, the observer, thefield-of-view or the angle-of-view causes a change in color coordinates(W. Hertog, “The design and implementation of a spectrally tunableLED-based light source: towards a new era of intelligent illumination”,PhD Thesis, Department of Optics and Optometry of the UniversitatPolitècnica de Catalunya, December 2016).

There are multiple potential causes for metamerism. Illuminant metamericfailure occurs when a change in the illuminant causes a difference inchromaticity between two items viewed under that light source. Observermetameric failure is caused by the difference in the visual systembetween 2 observers. Color perception among color normal observersvaries depending on pre-retinal filtering in the optical media (cornea,lens, and humors), macular photo pigment density, cone distributiondifferences, color neural processing differences, and differences incone spectral sensitivity. This cause of metamerism is underscored byreference to FIG. 5 depicting the variation in CMFs across multipleobservers.

Field-of-view metameric failure occurs when a stimulus is viewed withthe central fovea, due to a difference in concentration in cones, thecolor sensation is slightly different than when the same stimulus isregistered outside the central foveal region of the retina.

Angle-of-view metameric failure occurs depending on the gloss and othergonio-dependant characteristics of certain materials as the chromaticitychanges depending on the viewing angle.

Given control over the color reproduction environment in the currentlydisclosed method and system, the two forms of metamerism considered mostimportant are illuminant and observer metamerism. Approaches tomitigation of metamerism are addressed below in the DetailedDescription.

Color Gamut Limitations

Some article or product colors cannot be rendered on conventional RGBdisplays given that the article or product color spectrum residesoutside the gamut of the display. Also, printer gamuts are considerablysmaller than display gamuts; this is an inherent limitation in printcatalog representations of product colors.

PRIOR ART

Relevant prior art includes optical spectral sensors, color displays,and color matching methodologies.

Sensors

Optical instruments used to measure color include spectrometers andcolorimeters. Spectrometers measure the continuum spectrum of lightbeing sensed, whereas colorimeters typically are designed to output thelight intensity captured by RGB CMFs (Most often, standard CMFs areused.). Additionally relevant are spectral dispersion technologies usedin optical pulse compression.

In a spectrometer, light is either refracted or diffracted to spatiallydistribute the different wavelengths of a light source (whetherreflected or emitted) across a detector array, whereby the intensity oflight at a particular wavelength (or small spread of wavelengths) iscaptured on a single detector. In this way the continuum spectrum of thegiven light is measured. Spectrometric measurement is divorced from theissues surrounding human perception of color and any associatedambiguities (such as metamerism) because the entire color spectrum ismeasured. Colorimeters use calibrated illumination and color filtersthat mimic the spectral profile of human CMFs to provide three (RGB)integrated color values.

Colorimeters such as Color Muse, Nix Mini Color Sensor, and models byX-rite have been marketed to consumers for color matching applications.The following features are advertised for Color Muse:

-   -   Built in illumination source    -   Constant illumination and viewing angle    -   Constant “observer”    -   Elimination of area effect and contrast effect    -   Color difference measurement

However, in the present application, spectrometers are the favored colormeasurement device in order to avoid observer metamerism. High endspectrometers exhibit exquisite spectral resolution, but lower costdevices can be used to achieve spectral resolution on the order of ananometer.

Optical detectors of importance include integrated multi-spectralsensors, most notably, those vended by Austria Micro Systems (AMS)(previously manufactured by MAZeT GmbH). These sensors are fabricatedfrom multiple dielectric filters rendered in a single miniature packagewith electronic interface. Such devices are useful for closed-loopwavelength control of LEDs.

Displays

Color display displays include commercial solid state devices such asthose associated with smartphones, tablet computers, and monitors forcomputer, entertainment, and industrial applications. Also, luminairetechnology used for colored light illumination is relevant. Importantconsiderations are the number and saturation level of the primaries usedin the display as this will determine the color gamut that can bedisplayed. The core LED technology underpinning many of these displaydevices is of paramount importance. Among critical LED parameters areoptical bandwidths, wavelength availability and stability withtemperature and drive current, and flux levels. Significant performanceimprovements in LED and associated LCD technology have occurred inrecent years.

Color Matching Methods

Prior art additive color matching methods are most relevant to thepresent application given the emphasis on active display of reproducedcolor. In this context, the many variants of color monitor calibrationused in work flow protocols within the graphic arts and publishingindustries are important. Many of these methods involve software hostedon monitors that is used in concert with colorimeters or spectrometers.Characterization of color reproduction devices is achieved with deviceprofiles; exemplary is U.S. Pat. No. 8,246,408.

The most widely used profiles are those of the International ColorConsortium (ICC). These permit correct color reproduction when imagesare input from a scanner or camera and displayed on a monitor orprinted. They define the relationship between the digitalrepresentations of color information the device receives or transmitsand a standard color space defined by ICC and based on a measurementsystem defined internationally by CIE. Thus, a profile can be availablefor a scanner, camera, display and printer; the fact that they refer toa standard color space permits their combination in a workflow so thatthe correct color is maintained from imaging to display or printing.

An ICC profile is one that conforms to the ICC specification. Byconforming to this specification profiles may be exchanged and correctlyinterpreted by other users. The two main types of profiles are source(input) and destination (output) profiles and essentially consist oftables of data that relate the device chromaticity co-ordinates to thoseof the standard color space defined by ICC. There are variousrelationships defined in each profile (known as rendering intents).Special types of profiles (devicelink, and abstract) are defined forspecial workflow applications.

Metamerism Reduction

Various prior art methods of reducing observer metamerism can be cited,among these include increasing the bandwidth of primaries, selectingspecific red, green, and blue wavelengths, use of more than threeprimaries, and a method for observer-dependent color imaging wherein thecolor workflow is tuned to match one of several observer classes. In thelatter case, means to assign an observer to such classes can bephysiologically based. Noteworthy is U. S. Patent ApplicationPublication Number 20140028698 which discloses applying a metamerismcorrection transform to a input color image to determine an output colorimage in an output color space appropriate for display on the colordisplay device, the output color image having a plurality of outputcolor channels, each of the output color channels being associated withone of the device color primaries, wherein the metamerism correctiontransform modifies colorimetry associated with the input colors toprovide output color values such that an average observer metamericfailure is reduced for a distribution of target observers.

SUMMARY OF THE INVENTION

There are well developed technologies that can be used to specify colorsin quantitative fashion and reproduce such colors by active displaymeans. For color measurement and quantification, spectrometers orcolorimeters can generate a quantitative, reproducible description ofany particular color when observed under controlled illumination. In thespectrally accurate display of color, illumination sources such as LEDsand OLEDs can provide display primaries for additive color synthesisthat can be wavelength controlled.

What is needed is a viable, cost effective, convenient method for remotecolor reproduction and matching that can be used by product vendors andconsumers as well as other parties interested in high fidelity remotereproduction of color. More particularly, to be sought is a method andsystem that permits color identification and matching for articles orproducts that are not locally observable by an interested party.Presently disclosed is a business method and system to achieve theseobjectives. The method involves actions taken by both a first party,such as a product vendor and a second party, a user remote from thefirst party, such as a consumer. In this method, the vendor will use asensor to capture product spectral color information under controlledillumination conditions. This spectral information would be communicatedwith prospective consumers. Such information can be digitized and codedfor publication online or in printed material associated with the givenproduct. More particularly, spectral information can be published on awebsite that can be electronically downloaded then uploaded into adisplay device or manually entered in a display device. This spectralinformation can be coded for data compression. In the case of electronictransfer of data, a compressed data file can be uploaded into thedisplay device and decoded for use by the display device. The data alsocan be emailed for such use.

The remote user (ex. consumer) either would upload this information intoa compact display that would provide a high fidelity rendering of theactual article or product color or could use a display (smartphone,tablet, or monitor) calibrated to the remote user's visual response.Variations on this method include different approaches to mitigateobserver metamerism which otherwise would cause failure to render colorswith fidelity adequate that the typical consumer or other remote userwould consider the color rendition matches the original color.

A first such approach to diminish observer metamerism comprises thecalibration of displays to be used by a consumer for product colorreproduction. Such calibration would be performed against measured userCMFs. A second approach makes use of a multi-primary display exhibitingspectral match to the product spectrum under colorimetric constraints.The colorimetric constraint comprises either a match to standard CMFs,such as CIE 1931 CMFs, or to the measured remote user's CMFs. A thirdapproach, which totally circumvents metamerism, utilizes an adaptivespectral filter to filter the user's ambient light with the reflectancespectrum of the colored article in question. Hereinafter, the use of theterminology “reflectance spectrum” refers to the reflectance spectrum ofan article or product.

In the system to support implementation of these methods, a spectralsensor would be employed by the vendor of the colored product in theform of a colorimeter or spectrometer with digital output, and aportable display would be used by the consumer to render the productspectral color information published by the vendor.

The consumer display can be a smartphone, color tablet, color monitor,or a compact, handheld monocular or binocular device, after the fashionof a virtual reality headset. Different embodiments of this latterdevice use a) either several illumination primaries (multiprimaries) toreproduce the color spectrum of the product or b) spectrally modulateambient light with the article or product spectrum. In the multiprimaryimplementation, blocking ambient light would eliminate color perceptionissues associated with ambient and background light. Additionally, asstated, some embodiments of the method require measurement of theconsumer's CMFs. The functionality to perform such measurements can beinstantiated as a standalone compact portable device or can beincorporated into the aforementioned monocular or binocular displaydevice.

Below is a lexicon of terms used in this disclosure to support themeaning of the specification and to clarify interpretation of theappended claims.

Definitions

Average human observer—refers to a human observer with no particularvisual handicap.

Colorimetric matching constraint—when optimizing a match of the articleor product spectrum to the light from a combination of multi-wavelengthprimaries, this constraint is applied to also drive a best match to theoutputs from CMFs, either average observer CMFs or the measured consumerCMFs (a consumer CMF colorimetric matching constraint).

Color associated with the composite spectrum—the color that is producedby a display using mixing ratios for the primaries that have beencalculated to generate a match to an article or product compositespectrum.

Color checker—is an array of scientifically prepared colored squares ina wide range of colors that span the visible spectrum and that representthe range of natural objects encountered in the world—when placed in ascene they can be used to color calibrate display of the photographedscene on any given display.

Color mixer—a device which combines radiation from sources havingdifferent center wavelengths so as to create a light field of spatiallyuniform color.

Composite spectrum—the spectrum resulting from thewavelength-by-wavelength multiplication of the article or productreflectance spectrum and an illumination spectrum—the illuminationspectrum could be that used to illuminate the actual article or productor in some applications, it can be the consumer's lighting spectrum.

Consumer—a human user who is interested in identifying the color of aproduct for sale online or in printed pictures as they would perceivethe physical item, also refers to a consumer of such color informationwishing to establish the true color of the product.

Consumer's or remote user's illumination spectrum—the spectrum ofambient or directed light used to illuminate articles in the consumer'sor remote user's environment.

Custom RGB display—a handheld display useful for displaying color usingRGB primaries.

Display in high fidelity—a quality of remote color display thatreproduces the article or product color such that normal observers theaverage human observer would consider that the reproduced color matchesthe original color of the article or product.

Display or display device—a device capable of displaying at least onepixel of color. The device can be an active light emitting device as inthe case of a multi primary display using color emitters like LEDs or apassive screen that may be frosted for display of spectrally-filteredambient light.

Ensemble of advertised products—a sample of products that would beadvertised online or in video or print media sufficiently large torepresent the gamut of colors that need to be reproduced for consumers.

Illumination type—one of the standard illumination spectra, such as D65.

Local—characterizing an item that can be viewed directly by the partyinterested in the item's color.

Multiprimary display—a color display using multiple primary wavelengthemitters.

Pattern color map—a spatial mapping of the color code descriptors thatcompose a color pattern and can be used to display the color pattern.

Primaries—the set of three or more disparate wavelength optical sourcesused to compose a given color.

Published spectral information in a form for consumer use—eitherspectral data or corresponding primary mixing levels for remote user(ex. Consumer) use in display of article or product color in the form ofa) electronic data that can be used in a color display device to displaythe reproduced color, b) printed form that can be manually entered intoa display device, c) data published on a website that can beelectronically downloaded for use in a display device or manuallyentered in a display device, or d) emailed data that likewise can beused for display of the article or product color.

Reflectance spectrum—comprises the values of the normal reflectance ofthe article or product as a function of wavelength, hence this spectrumcharacterizes the color of the article or product independent ofillumination.

Remote—characterizing an item that is not local to the party interestedin observing the item's color.

Remote reproduction—high fidelity reproduction of an article or productcolor at a location remote to the article or product.

Remote user—a human who either is a consumer as per the above definitionof consumer or is a person making use of the presently disclosed methodor device to reproduce the color of an article that is remote to them.

Remote user's ambient light—either the light in the environment of theuser or light selected by the remote user to be directed to the colorreproduction device for combination with the article or productreflectance spectrum in creation of the reproduced color.

RGB—red, blue, green.

Single pixel and single pixel data—refers to the single color of thearticle or product and the single colors of the color checker colors,not associated with an image. A single pixel of color can be expanded tomultiple pixels in color calibration software for human visualinteraction and still be considered “single pixel” to emphasize thenon-imaging nature of the color reproduction or matching method.

Spatial light modulator—any device that is used to spatially modulate alight in one or more dimensions. The modulation can be accomplished withliquid crystal, magnetoptical, or digital micromirror devices.

Spectral information—information about an article or product color thatcomprises actual spectra, associated tristimulus values, amplitudes of aset of multi-primary LED intensities optimized to match the articlespectrum, encoded spectral information, and ambient or illuminationspectra.

Spectral match of the multi-primary display—calculating the mixingratios for the LEDs of a multi-primary display to optimally match acolor spectrum.

Standard color matching functions (CMFs)—any one of a number of standardCMFs for normal observers such as CIE 1931 or CIE 1964.

Total spectrum—this is the same as a composite spectrum.

True color—color rendering of the article or product that exhibits aspectrum that is sufficiently close to the actual article or productspectrum (reflectance spectrum or total spectrum) that the average humanobserver cannot discern a difference between the rendered spectrum (truecolor) and the article or product spectrum.

User—a person(s) who would be in receipt of an article or product colordescription code or information, wishing to view the associated color ofthe article or product.

Vendor—an entity selling a given product online or through printeddescription, also refers to a provider of color information about thegiven product.

DESCRIPTION OF THE FIGURES

FIG. 1 is a plot of the CIE chromaticity diagram.

FIG. 2 is a plot of a three-primary gamut on the CIE chromaticitydiagram.

FIG. 3 is a plot of a five-primary gamut on the CIE chromaticitydiagram.

FIG. 4 is a plot of the three CIE CMFs.

FIG. 5 is a plot of the spread in RGB CMFs among multiple standardobservers.

FIG. 6 is a plot of MacAdam ellipses on the CIE chromaticity diagram.

FIG. 7 is a plot of the just noticeable wavelength difference across thevisible spectrum.

FIG. 8 is the spectral plot of primaries at eight specific wavelengthsthat can be used to minimize observer metamerism.

FIG. 9 is a pictorial view of a color reproduction display device forconsumer use.

FIG. 10 is schematic and functional diagram of the control circuitry fora multi-primary display used for color reproduction.

FIG. 11 is a schematic and functional diagram of a color reproductiondisplay system including color matching function measurement capability.

FIG. 12 is a functional block diagram of a color reproduction displaydevice that employs a multiprimary filter.

FIG. 13 is a functional block diagram of a color reproduction displaydevice that employs a spectrum disperser and spectrum combiner with aspatial light modulator.

FIG. 14 is a schematic diagram of a color reproduction display devicecorresponding to the block diagram of FIG. 13 that employs 4 gratings.

FIG. 15 is a functional block diagram of a color reproduction displaydevice that employs a combined spectrum disperser—spectrum combiner witha spatial light modulator.

FIG. 16 is a schematic diagram of a color reproduction display devicecorresponding to the block diagram of FIG. 15 that employs 2 gratings.

FIG. 17 is a schematic diagram of the color reproduction display deviceof FIG. 16 but with the inclusion of an optical isolator.

FIG. 18 is a schematic diagram of a color reproduction display devicecorresponding to the block diagram of FIG. 13 that employs 4 prisms.

FIG. 19 is a schematic diagram of a color reproduction display devicecorresponding to the block diagram of FIG. 15 that employs 2 prisms.

FIG. 20 is a schematic diagram of the color reproduction display deviceof FIG. 19 but with the inclusion of an optical isolator.

FIG. 21 is a pictorial diagram of a color reproduction display devicecorresponding to the block diagram of FIG. 15 that employs a singleprism.

FIG. 22A is pictorial diagram of a VIPA used in concert with a grating.

FIG. 22B is a two-dimensional map of the light dispersed by theVIPA-grating combination.

FIG. 23 is a pictorial diagram of a color reproduction display deviceemploying a VIPA and grating.

FIG. 24 is a pictorial diagram of color reproduction display deviceemploying a VIPA and GRISM.

DETAILED DESCRIPTION OF THE INVENTION

The method and system of the present disclosure requires the functionsof article or product spectral measurement, for a multiprimary display,the calculation of mixing ratios for the primaries of the display toreproduce the color of the article or product, and implementation ofspatial color mixing of the display primaries and for a spectralfiltering display, the use of article or product reflectance spectra tofilter ambient light.

Measuring the Spectrum of the Article or Product Color

There are nuances surrounding the capture of article or product colorspectral information, some of these issues are avoided in the case ofnarrow field of view light capture, without background reflections.Nevertheless, the diffuse and specular components of article or productreflectance must be addressed. The spectrum capture should be along anormal to the local surface of the article or product and there shouldbe not shadowing due to article or product geometry.

There are two modes of article or product spectrum measurement in thepresent color reproduction method. In the first mode, the total articleor product spectrum is measured by the product vendor to include theresult of both illumination and reflection. In the second mode, thereare two instances of illumination spectrum measurement: a) theillumination spectrum is specially measured by the article or productvendor or an adopted standard is adequately implemented, and b) theillumination spectrum in the environment of the consumer is measured.This second mode is intended for use in mitigating illuminantmetamerism.

The table below summarizes standard illumination spectra. The prevailingindustry guidance is that CIE standard illuminant D65 should be used inall colorimetric calculations requiring representative daylight. It isadvisable that at least two other instances of illuminants be used,perhaps one each for incandescent light and fluorescent light,respectively. Which illuminants to use should become an industrystandard for the present method.

CIE Standard Illuminants Description First three standard illuminants -introduced in 1931 A Incandescent light with a correlated colortemperature of 2856 K B Representative of noon sunlight, with acorrelated color temperature of 4874 K C Average daylight (not includingultraviolet wavelength region) with a correlated color temperature of6774 K D series (Natural Daylight) D50 Representation of a phase ofdaylight at a correlated color temperature of 5000 K D55 Representationof a phase of daylight at a correlated color temperature ofapproximately 5500 K D65 Intended to represent average daylight and hasa correlated color temperature of approximately 6500 K F series(Fluorescent Lighting) F1-F6 Spectra for “standard” fluorescent lampsconsisting of two semi-broadband emissions of antimony and manganeseactivations in calcium halophosphate phosphor F7-F9 “Broadband”(full-spectrum light) fluorescent lamps with multiple phosphors, andhigher CRIs F10-F12 Narrow triband illuminants consisting of three“narrowband” emissions (caused by ternary compositions of rare-earthphosphors) in the R,G,B regions of the visible spectrumFirst Mode of Article or Product Spectrum Measurement

Having measured the article or product color spectrum that includesillumination and article or product reflectance, the vendor can publishspectrum information corresponding to three different illuminants. Then,the consumer will be able to reproduce the article or product color asviewed under these three different lighting conditions.

Second Mode of Article or Product Spectrum Measurement

In this mode, the article or product color spectrum (that includesillumination and article or product reflectance) and the illuminationspectrum are measured by the vendor. If a standard illuminant isadequately emulated by the vendor, then the identity of the standardspectrum can be published by the vendor for use by the consumer. Sincethe total spectrum comprises the product of the illuminant amplitude andreflectance amplitude at each wavelength, the reflectance spectrum canbe derived.

The problem of separating illumination and reflectance spectra has beenaddressed in image and machine vision applications, which involvepixel-by-pixel separations. This has included the issue of spatiallynon-uniform illumination. (Xiaochuan Chen, Mark S. Drew, and Ze-Nian Li,“Illumination and Reflectance Spectra Separation of Hyperspectral ImageData under Multiple Illumination Conditions”, Electronic Imaging 2017:Color Imaging XXII, Displaying, Processing, Hardcopy, and Applications,29 Jan.-2 Feb. 2017, San Francisco.) Hence, a host of prior artalgorithmic approaches to addressing this problem exist. Fortunately,the present application largely involves the degenerate case of uniformillumination and a scalar (single pixel) color signal. Prior art offersa number of ways to optimally estimate the reflectance at wavelengthswhere the total spectrum signal-to-noise-ratio is poor. For an articleor product of uniform color, light from only a small region of thearticle or product surface needs to undergo spectral measurement. In thecase of articles or products exhibiting variable color, uniform colorregions of the article or product should be independently measured.

In order to reproduce the article or product color as would be observedin the consumer's environment with an multiprimary display, the ambientlight or illumination spectrum present in the consumer's environmentmust be measured. Then it can be multiplied by the reflectance spectrapublished by the vendor to create the total spectrum that would beobserved in this environment. Hence, there is need for a low costspectrometer that would be used by the consumer in the presentlydisclosed method and system. Fortunately, do-it-yourself spectrometerswith sub nanometer wavelength resolution (able to separate the Sodium-Dlines) can be made very inexpensively. Examples use gratings comprisingDVD material or grating films and a webcam detector. This technology canbe incorporated into the color reproduction display device discussedbelow.

Third Mode of Article or Product Spectrum Measurement

In this mode, again, only the reflectance spectrum of the article orproduct is measured by the vendor, but the user's ambient light spectrumis not measured. The reflectance spectrum will be used with an adaptivespectral filter to spectrally shape the user's ambient light inaccordance with the measured reflectance spectrum.

Lighting Conditions

As is well known in the prior art associated with article or productphotography, guidance exists for optimal color photography of articlesor products to include approaches to the use of fill or bounce light tosoften shadows and choice of surrounding illumination environment.Emphasis in the presently disclosed method and system is to capture asmall field of view that does not exhibit shadowing. However, someconvex surfaces and textures may require such attention.

Generation of Amplitudes for a Multi-Primary Display

As discussed below, one approach to minimizing observer metamerisminvolves use of a multi-primary display with LED wavelengths determinedby optimization calculations. To determine the relative intensities ofthese LEDs that best match the measured article or product spectrum, themethod of Murakami et al (Yuri Murakami, Jun-ichiro Ishii, Takashi Obi,Masahiro Yamaguchi, Nagaaki Ohyama, “Color conversion method formulti-primary display for spectral color reproduction”, J ELECTRONIMAGING, vol. 13, 30 Sep. 2004, pp. 701-708.) is employed.

The method gives the amplitude values of each primary of a multi-primarydisplay device that minimize the spectral approximation error under theconstraints of tristimulus match. The constraint used in the conversionis a tristimulus match for the standard observer, which is the sameconstraint for the conventional color reproduction. Under thisconstraint, this method does not need any information about theindividual CMFs or deviations to minimize the difference between thespectra of the original object and the reproduced light.

If the color generation of an N-primary display is based on the additivemixture of the primaries, the spectral intensity of the reproduced lightP(λ) is approximately represented by

${{P(\lambda)} = {\sum\limits_{j = 1}^{N}{\alpha_{j}{p_{j}(\lambda)}}}},$where p_(j)(λ) (j=1, . . . , N) is the spectral intensity of thefull-emitted jth primary light and α_(j)(0≤α_(j)≤1) is the amplitude ofthe jth primary. If S(λ) is the spectral intensity reflected from thearticle for which color reproduction is desired, then the square errorbetween S(λ) and the reproduced spectrum by the N-primary display isdefined asE=∫[S(λ)−P(λ)]² dλ.

The method determines the set of primary amplitudes {α₁, . . . , α_(N)}that minimizes E. When minimizing E, the constraints that thetristimulus values of the CIE standard observer are accuratelyreproduced are imposed. That is∫t _(k)(λ)S(λ)dλ=∫t _(k)(λ)P(λ)dλ,k=X,Y,Z

Where t_(k)(λ) are the CMFs of the CIE standard observer. Theseconstraints are introduced because of the following reasons. If a set ofprimary amplitudes is optimized only for spectral approximation, thetristimulus errors for most observers can be considerably large,especially when the number of the primaries is insufficient. To reducethe average mismatch, tristimulus match for the CIE standard observer iseffective because CIE standard CMFs are designed to represent theaverage color matching response of the population of human observers.The algorithmic solution to this optimization problem is found in theabove reference to Murikami et al, which is incorporated herein byreference. Software to calculate the optimization solution is hosted ona computing platform for the vendor. These optimization results, in theform of relative LED amplitudes for either article or product totalspectrum or reflection spectrum, can be published by the vendor forconsumer use in the corresponding multi-primary display.

Color Mixing Optics

The presently disclosed method and system require means to create auniform color display from a plurality of LEDs of different wavelengths.The uniformity of such color mixing must be sufficient that colorvariation is not detectable within the observer's field of view.

Great impetus for achieving good color homogeneity in multi-wavelengthlight mixing comes from the commercial lighting industry and luminaireproduct development. Initial approaches to color mixing from multipleLED sources simply relied upon use of textured surfaces, or diffusersfor spreading of light. Often expensive optics with high numericalapertures are required to collect the spread light. Further, theefficiency and performance of such systems are inferior to newerapproaches that involve light guiding. These latter designs typicallyhave been optimized by simulation with Zemax or similar optical modelingsoftware.

Many patents have been issued on the subject of color mixing andhomogenization for LED sources. U.S. Pat. No. 9,746,596 is exemplary ofmethods that use molded optics and light pipe geometries. Also,commercially available optics have been developed for LED color mixing.Examples include high efficiency molded polymer lenses for RGBW LEDcolor mixing from Khatod, Milano, Italy (part number PL1590ME).

The most effective and compact implementations of color mixers thatachieve spatially homogeneous color and intensity use a combination oflight pipes, refractive and reflective interface geometries, anddiffusion. An example (Sun, C. C.; Moreno, I.; Lo, Y. C.; Chiu, B. C.;Chien, W. T. Collimating lamp with well color mixing of red/green/blueLEDs. Opt. Express 2012, 20, A75-A84) is a compact optical system forRGB color mixing that demonstrates use of only compact monotonicsurfaces in the optical design. It comprises a relatively short (lessthan 10 millimeters length), straight lightpipe with silver scattersheet reflective walls, a volume scattering diffuser, and a totalinternal reflection (TIR) output lens. A luminaire design for a largernumber of multi-wavelength LEDs (Maumita Chakrabarti, Henrik ChrestenPedersen, Paul Michael Petersen, Christian Poulsen, Peter BehrensdorffPoulsen, Carsten Dam-Hansen, “High-flux focusable color-tunable andefficient white-light-emitting diode light engine for stage lighting”,Optical Engineering 55 (8), August, 2016.) demonstrates a departure fromcolor uniformity over a few degrees viewing angle of less than 0.001percent. It exploits a microlens array, a parabolic reflecting surface,and a TIR lens.

Another approach which entails using freeform optics to map out lightray trajectories is exemplified by the design of Chen et al. (EnguoChen, Rengmao Wu, Tailiang Guo, “Design a freeform microlens arraymodule for any arbitrary-shape collimated beam shaping and colormixing”, Optics Communications, Volume 321, 15 Jun. 2014, Pages 78-85.)This freeform microlens array module, which shows better flexibility andpracticality than the regular designs, can be used not only to reshapeany arbitrary-shape collimated beam (or a collimated beam integratedwith several sub-collimated beams), but also most importantly to achievecolor mixing.

A novel mixing approach is detailed in U.S. Pat. No. 9,022,598 whichdiscloses combining the zero spatial frequency components of coloredsources to achieve homogenization of composite color. The inventionexploits the fact that extended and non-overlapping light emittingsources arranged in a specific pattern may overlap in Fourier space.

Finally, multi-wavelength beam combining can be achieved withconsecutive introduction of each color beam into the composite beamusing multiple dichroic filters like the LaserMUX™ filters manufacturedby Semrock. However, this approach is relatively expensive.

Light guiding techniques are most adaptable to the display concept ofthe present disclosure and support the fabrication of a compact,handheld display device as described in more detail below. In fact, thesame method of LED color mixing can be used for both rendering of colorsnecessary for measurement of consumer CMFs and in the finalmulti-primary display of reproduced article or product color.

A Preferred Implementation of Color Mixing

An adaptation of the aforementioned concepts that permits color mixingof as many as 8 different wavelength LEDs in a compact geometry canemploy light guiding, with volumetric and surface scattering, andappropriately designed refraction and reflection to obtain color displaythat exhibits imperceptible nonuniformity.

Measuring Individual CMFs

Fedutina et al. (M. Fedutina, A. Sarkar, P. Urban, P. Morvan, “(How) Doobserver categories based on CMFs affect the perception of small colordifferences?”, Color and Imaging Conference 2011 (1), pp. 2-7.)demonstrated in nine categories of observers based on color perceptionmetrics, significant departure of the individual response from the CIEstandard observer. The determination an individual's CMFs can beparamount in achieving color matches below the threshold of differencedetection.

Methods of Measuring CMFs

Various methods of measuring the consumer's CMFs delineated herein arewithin the scope of the present invention. The most commonly usedapproach is the maximum saturation method, which was used by Wright(Wright, W. D., “A re-determination of the trichromatic coefficients ofthe spectral colours”, Transactions of the Optical Society. 30:141-164,1929.) and Guild (Guild, J. 1932. The colorimetric properties of thespectrum. Philosophical Transactions of the Royal Society of London,Series A. 230:149-187.) to obtain color matches that were subsequentlyused to generate the CIE 1931 CMFs. In this method, the observer ispresented with a half field illuminated by a “test” light of variablewavelength, A, and a second half field illuminated by a mixture of red(R), green (G) and blue (B) primary lights. At each A, the observeradjusts the intensities of the three primary lights, so that the testfield is perfectly matched by the mixture of primary lights.

In Maxwell's method (referenced below), preferred for the presentapplication, the matched fields always appear white, so that at thematch point, the eye is always in the same state of adaptation whateverthe test wavelength (in contrast to the maximum saturation method inwhich the chromaticity of the match varies with wavelength). In amatching experiment, the subject is first presented with a whitestandard half field, and is asked to match it with the three primarylights. The test light then replaces the primary light to which it ismost similar and the match is repeated.

Fitting Data to Parametric Models

In the work of Asano et al. (Yuta Asano, Mark D. Fairchild, and LaurentBlondé, “Individual Colorimetric Observer Model”, PLoS One. Feb. 10,2016; 11(2):e0145671, eCollection), eight additional physiologicalparameters are added to the two parameters in the CIE 2006 PhysiologicalObserver construct to model individual color-normal observers. Theseeight parameters control lens pigment density, macular pigment density,optical densities of L-, M-, and S-cone photopigments, and λ_(max)shifts of L-, M-, and S-cone photopigments. By identifying thevariability of each physiological parameter, the model can simulate CMFsamong color-normal populations using Monte Carlo simulation which iscomputationally intensive.

Hardware Approaches to Measurement of CMFs

A concept demonstrated in 1989 was a visual four-channel colorimeterthat uses the Maxwell method (Mark Fairchild, “A novel method for thedetermination of CMFs”, Color Research & Application 14(3), June 1989,pp. 122-130.). It used laser lines for the three red, green, and blueprimaries and a broadband spectral source comprising a tungsten-halogenlamp. The three primaries plus the spectral source illuminated one halfof a bipartite field. The other half was illuminated with a daylightsimulator. The three primaries were intensity modulated by acousto-opticmodulators under observer control. Observers made matches using theMaxwell method for five wavelengths and simulated daylight. From thevisual results, color matching function for the entire visible spectrumwere estimated using a statistical model. The model assumed that CMFsare a linear transform of cone sensitivities convolved with differencesin the macular pigment and amount of scattering in the crystalline lens.The five wavelengths were selected to provide estimates of the level ofmacular pigmentation, the level of lens scattering, and the elements inthe linear transform. Nonlinear optimization was used to estimate themodel parameters. This approach can be revisited with an LEDimplementation for the presently disclosed method and system.

With time, advances in color matching filter measurement have providedsimpler, more compact, and cost-effective devices. Two foremost examplescomprise devices that also use Maxwell's method. In the first example(Yasuki Yamauchi, Yasuhisa Nakano, Masatomo Kamata, Katsunori Okajima,Keiji Uchikawa, Yuri Murakami, Masahiro Yamaguchi, and Nagaaki Ohyama,“Measurement of CMFs using a digital micro-mirror device”, OSA FallVision Meeting, December 2003.) the system can present a test stimuluswhose spectral power distribution can be arbitrarily set by adjustingthe power of every monochromatic light between 400 to 700 nm with a stepof 10 nm. This is realized by selectively switching a digitalmicro-mirror, on which the spectrally decomposed light from adiffraction grating is focused. Thirty two independent compound lightsare used as a test stimulus. The observer adjusts the color of the teststimulus to match that of the reference white. A two-degree bipartitefield is used to present the test and the reference stimuli.

A conventional bipartite apparatus to measure CMFs usually consists ofplural optical paths; a path for a test stimulus consisted of threeprimaries, and that for the reference stimulus. The primaries should bepresented to both optical paths, as “negative” light in the referencestimulus is sometimes required to complete color matching. Thus, theconventional apparatus should have plural light sources in each opticalpath and requires complicated alignments. In the second device example(Yasuki Yamauchi; Minoru Suzuki, Taka-aki Suzuk, Katsunori Okajima,“Measurement of CMFs with a compact and simple apparatus using LEDs”,OSA Fall Vision Meeting, December 2010.), so as to realize a compactapparatus to measure CMFs, the researchers developed a bipartiteapparatus with time-controlled LED lights.

Specifically, they used a single light source, which consisted of pluralLEDs inserted to a small integrating sphere (4″ diameter). A beamsplitter was used to divide the light into two optical paths. Theoptical path, which was delivered to a subject, was temporally switchedin alternating fashion. Its frequency was high enough for the observernot to detect the flicker of the lights. By changing the switch timingof the LEDs, it was possible to arbitrarily select any combinations ofthe LEDs to present in either the test or the reference stimulus area.Subjects adjusted the intensity of the test stimulus which wascontrolled by pulse width modulation. The resulting device was a compactCMF-measuring apparatus that can present bipartite stimulus with asingle light source by time-controlled switching and modulation of theLEDs.

An embodiment of the presently disclosed method and system involvesincorporating CMF-measuring functionality. In one approach, theindividual's CMFs are measured with the same device that is used todisplay a reproduced article or product or article color. The same typeof LED light collection and mixing optics are used for both CMFmeasurement and reproduced article or product color display. Also, it isimportant to emphasize that a consumer need measure his CMFs only once.

Implicit Measurement of CMFs

An embodiment of the presently disclosed method and system thatimplicitly incorporates consumer CMF information comprises vendor use ofcolor calibrating color checkers and consumer use of a softwareapplication that exploits the color checker information for displaycolor calibration to compensate for illumination and camera spectraleffects. The display however needs to have calibration to spectralstandards such as by use of a colorimeter before shifting its colorresponse using a color checker. The aforementioned do-it-yourselfspectrometer can be modified to be a tristimulus colorimeter that usesthe CIE CMFs to filter raw spectra. For this, the CMFs are used insoftware to digitally filter the spectral data.

A popular color checker product from X-rite has the followingdescription from their website (X-ritephoto.com):

“The ColorChecker® 24 Patch Classic target is an array of 24scientifically prepared natural, chromatic, primary and gray scalecolored squares in a wide range of colors. Many of the squares representnatural objects, such as human skin, foliage and blue sky. Since theyexemplify the color of their counterparts and reflect light the same wayin all parts of the visible spectrum, the squares will match the colorsof representative sample natural objects under any illumination, andwith any color reproduction process.”

The X-rite ColorChecker Passport product suite includes three differentcolor patch arrays that are placed in the scene to be photographed. Anassociated software application uses scene images containing these colorpatch arrays to calibrate the photo display. This technology can beemployed in the presently disclosed method and system in the followingways. In the first way, one or more images of the article or product arecaptured with the color checker patch arrays included in the image (Anindustry agreed-upon standard for illumination would be desirable.) Suchimages, preferably in an electronic form (likely involving conversionbetween DNG format and others) would be used by the consumer to colorcalibrate the consumer's display (smartphone, tablet, monitor, etc.) forcorrect article or product color reproduction by use of an automatedsoftware application.

A custom color checker array of colors can be composed based on ananticipated gamut of colors spanned by a large ensemble of articles orproducts because many colors within this gamut may be more saturatedthan those of the natural environment. It may be necessary for creationof an industry standard as a result. In the example of wound imaging, itwas demonstrated that choosing a custom array of colors that bestrepresented the wound images in a database improved color rendition uponreproduction compared to the standard Macbeth color checker (HazemWannous, Sylvie Treuillet, Yves Lucas, Alamin Mansouri, Yvon Voisin,“Design of a Customized Pattern for Improving Color Constancy AcrossCamera and Illumination Changes”, Conference: VISAPP 2010—Proceedings ofthe Fifth International Conference on Computer Vision Theory andApplications, Angers, France, May 17-21, 2010—Volume 1)

Another way to employ this technology is to focus on “single pixel’information, since a main application of the presently disclosed methodand system is single color capture and reproduction. In this case, it isnecessary only to include single pixels of the colors of the colorchecker captured under the same lighting as the article or product (thearticle or product color also may be represented by a single pixel).These single pixel values would be published for use by the softwareapplication for the consumer's display color calibration.

One final prospect for effectively measuring a consumer's CMFs involvesusing a smartphone with the display in camera viewfinder mode. An appwould permit the user to adjust the viewfinder display hue(s) to matchthe hue(s) of the actual object, color checker, or scene being viewedthrough the camera. Given the spectral responsivity of the camera, theuser CMFs can be determined. Other software functionality would importthe vendor-provided article or product spectrum information and filterit with the user CMFs to display the resulting reproduced article orproduct color on the smartphone display.

A Preferred Implementation of CMF Measurement

The approach to measurement of the consumer's CMFs favored in thepresently disclosed method and system uses a time division multiplexeddisplay of each bipartite field using the same LEDs, as discussed byYamauchi et al. In this approach, a beam splitter splits the color mixedlight into two optical paths; a test stimulus path and a referencestimulus path. Each optical path is alternately blocked off by anoptical chopper. Depending on the timing of the optical chopper, onlyone of the test or the reference stimulus is presented to the observer.Moreover, the switching timing of the LEDs is controlled to synchronizewith that of the optical paths. Therefore, it is possible to arbitrarilychoose any combinations of the LEDs to be presented both to the test andto the reference stimulus area. A switching frequency of 100 Hzpermitted the perception of a continuous stimulus.

For the presently disclosed concept, the optical chopper (switchingfunction) can be accomplished by a low cost projector LCD operated as aspatial light modulator (SLM) that shutters each bipartite fieldindependently. The consumer would adjust the individual LED intensitiesthrough pulse width modulation (PWM) control (color weighting of LEDs isachieved in the multi-primary display by PWM also). Processing meansincluded in the CMF measuring device support Maxwell's method ofmeasurement.

The CMF measurement device can be standalone or preferably made part ofthe product color reproduction display. In the latter case, the CMFmeasurement LEDs can be a subset of those used in the multi-primarydisplay or additional LEDs exhibiting other wavelengths. In a monoculardisplay, the visual field is partitioned when the display device is inCMF measurement mode. For multi-primary display of reproduced productcolor, the full monocular field would not be partitioned.

Implementation of Article or Product Color Reproduction Display withMultiple Primaries

In a first example embodiment of the presently disclosed method andsystem, optimum choice of LEDs is paramount for achieving colorreproduction with adequate fidelity. LED performance parameters such asnominal intensity operating regime and current levels, relativewavelength insensitivity to ambient and junction temperature, andoptical bandwidth must be optimized for the present application. Whenused as primaries for color reproduction, narrowband LEDs permitincreased color display gamuts but can worsen metamerism, whereasbroadband LEDs, by filling in spectrum, can diminish metamerism at theexpense of more limited gamuts.

In the paper by Ramanath, (R. Ramanath, “Minimizing observer metamerismin display systems,” Color Research and Application, Vol. 34, pp.391-398, 2009), observer metameric failure for different types ofdisplays having three primaries is examined. In particular, Ramanathexplores the comparative occurrence of observer metameric failure amongdifferent electronic display devices, including cathode ray tube (CRT)displays, liquid crystal display (LCD), digital light processor (DLP)and LED based displays, a cold cathode fluorescent lamp (CCFL) baseddisplay, and a laser display. Ramanath concludes that observer metamericfailure can occur more frequently, and provide greater perceived colordifferences, as the display spectrum narrows (smaller FWHM) or thenumber of modes in the display spectrum increases. As a result, thelaser display and CCFL display, which lack spectral color diversity dueto narrow or multi-modal spectra, have a high propensity to causeobserver metameric failure. By comparison, the CRT and lamp based DLPdisplays, which have broad primaries (Δλ of approximately 60-70 nmFWHM), exhibit low potential for observer metameric failure. In the caseof laser displays, where the spectral bandwidths can easily be 2 nm orless in width, a small expansion of the lasing bandwidths, at the costof a small color gamut decrease, would provide a reasonable trade-off ifobserver metameric failure is significantly decreased. However, Ramanathfound that spectral distributions with moderate FWHM bandwidths (Δλ ofabout 28 nm), such as LED illuminated displays, can still producesignificant perceptible observer metameric failure, suggesting thatreductions in observer metameric failure may not come quickly withincreases in spectral bandwidth.

It is critical to reduce observer metamerism in any method that seekshigh fidelity color reproduction. As discussed previously, the presentmethod invokes one of two alternative approaches to mitigation ofobserver metamerism. In one approach, the CMFs of the consumer aremeasured so that an article or product spectrum rendered against theseconsumer CMFs in an RGB display creates color reproduction fidelity. Inthe alternative approach, the article or product spectrum ismathematically optimized for a multi-primary display using LEDwavelengths determined to reduce observer metamerism.

Further, there are two options for the former approach. In one, thevendor-published article or product spectrum is filtered by the measuredconsumer CMFs to provide drive signals to the red, green, and bluechannels of a custom LED display, using three (or multiples thereof)LEDs. In the other, extant displays such as those of a smartphone,tablet, or monitor are calibrated against the measured consumer CMFs. Alow cost spectrometer (of the DIY variety put into large scaleproduction or the low cost kit for a smartphone spectrometer) can beemployed for this purpose.

There are two options for the spectrum mathematical optimizationapproach, namely, varying relative intensities of individual primariesof a multi-primary display under a constraint that a) uses the CMFs ofCIE standard observers or b) uses the measured CMFs of the user. Thelocation of the computing platform that performs such optimizationswould be dictated by which source of CMFs (CIE standard observers or theuser) was used and the type of color reproduction application, whetherit is remote consumer color matching or another application.

Custom Color Display

A custom three color LED-based display will render colors in accordancewith the measured consumer's CMFs. Hence the wavelengths and opticalbandwidths of the RGB primaries are not critical with respect toobserver metamerism, but can be optimized for improved gamut. Thedisplay would be incorporated into a handheld unit after the fashion ofFIG. 9 .

The aforementioned latter approach to reducing observer metamerism isbased on some multi-primary display research (David Long, Mark D.Fairchild, “Reducing observer metamerism in wide-gamut multi-primarydisplays”, SPIE Proceedings Volume 9394, Human Vision and ElectronicImaging XX; 93940T (2015). It had been postulated that multi-primarydesign paradigms may hold value for simultaneously enhancing color gamutand reducing observer metamerism, considering expansion of the areaspanned on the chromaticity diagram and increased spectrum sampling.This research determined that by carefully selecting primary spectra insystems employing more than three emission channels, intentionalmetameric performance can be controlled. Different wavelength sourceswere used to minimize observer metamerism against the CIE standardobserver CMFs over an ensemble of reference spectra. The resulting 8Gaussian model primaries are provided in FIG. 8 . The identifiedwavelengths are 425, 450, 490, 524, 555, 595, 630, and 670 nm. Hence,the multi-primary display of the presently disclosed method and systememploy LEDs with wavelengths such as these derived from minimum observermetamerism optimizations. It is possible to further decrease the errorin spectrum matching by using more than 8 primaries.

An example display device for the presently disclosed method and systemis in the form of a headset 31 (similar to a virtual reality headset)which blocks ambient and background light as depicted in FIG. 9 . Thehousing 33 is compact but sufficient to house optics and electronics.The molded interior 35 is designed to fit the facial contour and therebyprevent ambient light entry into the visual field when the head strap 39is affixed to the head in snug fashion. Both left and right apertures 37admit reproduced color light or can be used to display sequences ofcolors for the purpose of CMF measurement. The LEDs, color mixingoptics, associated electronics and batteries are incorporated within.

Whether using a custom RGB three color LED display or an 8-primary LEDdisplay for variants of the presently disclosed method and system,attention must be paid to wavelength stability of the LEDs. As statedbefore, shifts in wavelengths approaching one to two nanometers areproblematic given this is the threshold of change detectable by humans.Consideration must be given to how the wavelengths of LEDs selected foruse in the custom RGB or multi-primary display can be made stationaryover variation in drive level and ambient temperature.

Color mixing ratios require variable intensity of the individual LEDs.The intensity of the LEDs is altered either by continuous current(analog) dimming or by pulse width modulation (PWM) of constant currentsources (ex. the integrated current source LT3083). Attempts to decoupleLED drive level (current) from wavelength shifts have emphasized thelatter approach. However, PWM does affect LED wavelength (StevenKeeping, “LED Color Shift Under PWM Dimming”,https://www.digikey.com/en/articles/techzone/2014/feb/led-color-shift-under-pwm-dimming).It turns out that the change in peak wavelength (and hence chromaticity)is due to the fact that lower duty cycles heat the LED p-n junction lessthan higher cycles. The physics is complex, but in essence, junctiontemperature alters the chromaticity because the LED's band gap (whichdetermines the wavelength of emitted photons) narrows as the temperaturerises. It is important to point out that LED wavelength shift due toaging is not a factor for the currently disclosed method and systembecause it takes thousands of hours before human observation woulddetect a change.

Given this state of affairs, remedies sought for tendencies to incurwavelength shift appear in the form of two approaches. In the first, theLED current nominally is set to correspond to the nominal targetwavelength and nominal intensity and PWM is used to precisely establishLED intensity to satisfy color mixing ratios. In this case, wavelengthsof the LEDs are sensed to provide feedback control of current drive,thereby maintaining constant wavelength.

FIG. 10 details a candidate wavelength and intensity control circuit.Intensity and wavelength detection control LED current, whereasintensity detection controls PWM duty cycle for the purpose of colormixing. The PWM duty cycle is set corresponding to the appropriate colormixing value and then the current is adjusted to tune the wavelength.There may be some iteration in adjustment of PWM and current to maintainLED target intensity and target center wavelength. A processor ormicrocontroller 51 receives inputs 79 from the user interface 77 thatcontrols the mode of operation, i.e. as a color reproduction display ora CMF instrument. In the color reproduction mode, the user interfacesupplies the target color amplitude weights for color mixing. Theprocessor 51 determines the appropriate duty cycle for the various pulsewidth modulators 63 to establish relative LED 59 intensities. Theintensities of the LEDs 59 are measured by detector 77 in concert withthe transimpedance amplifier 57 using feedback network 75. The analogoutput of this amplifier is converted by A to D converter 55 for inputto processor 51. The modulators 63 control the current switches 61. Theclosed-loop current drive level for each LED 59 is established byprocessor 51 which communicates the corresponding voltage levels toprogrammable voltage sources 67 based upon the responses 72 of the colorfilter chip {A57262) 71. The integration time for the filter outputs ofthe AS7262 is 5.6 milliseconds, so continuous color control cannot beperformed because the update rate would be below 400 Hz human perceptionflicker frequency. This implies intermittent closed-loop calibration ofcolor, but this should not be problematic given the slow rate of colordrift. In the CMF measurement mode, the processor 51 controls thedisplay of bipartite color matching sequences and records user responsesthrough the user interface in order to determine the user CMFs.

Different LED technologies, device geometries, and operating regimesexhibit different wavelength shift behavior with drive current andambient temperature. For example, some surface mount LEDs undergoinglarge current changes only change dominant wavelength by 2 nm, whereasambient temperature can shift wavelength +0.03 to 0.13 nm/degrees C.depending on die type. For the commercial temperature range of 0 to 70degrees C., this would result in a center wavelength shift of between2.1 and 9.1 nanometers. Also, the center wavelength andfull-width-half-max (FWHM) of the spectrum vary with forward current. Soclosed loop wavelength control by current variation for this category ofLEDs would be counterproductive. However, an LED with a small wavelengthsensitivity to junction temperature, dλ/dT, tends to have a smallwavelength sensitivity to forward current, dλ/dI_(F).

The work of Raypah et al. evaluated several manufacturers of low powersurface mount device (SMD) LEDs to determine that junction temperatureapproximately tracks ambient temperature at full forward current (MunaE. Raypah, Mutharasu Devarajan, and Fauziah Sulaiman, “Modeling Spectraof Low-Power SMD LEDs as a Function of Ambient Temperature”, IEEETransactions on Electron Devices, February 2017, pp (99): 1-7.). Forcategories of low power SMD devices, this implies that a commercialtemperature range swing results in the same junction temperature swingwhich makes low dλ/dT devices an acceptable paradigm. Hence, the bestapproach is to search out LEDs with small wavelength sensitivities tojunction temperature and current and use bin selection to get under 1 nmerror in initial peak wavelength. Then the system can be operated openloop with respect to LED wavelength control. PWM would be used inestablishing color mixing ratios.

Examples of wavelength stable LEDs are given in the table below.

Temperature Sensitivity LED Part (nm/deg C.) InGaN Mars Green LED Chippart no. 0.030 ES-CEGHM10A Seoul Semiconductor 801 Red Series 0.026 partno. SRT801-S/STR0A12AR LUXEON Rebel and LUXEON Rebel 0.01 to 0.05 ESColors InGaN Cree ® TR5050 ™ 0.048

Another consideration is to use multiple LEDs of the same wavelength foreach of the primaries. This reduces drive current to any given LED bythis same multiple, thereby reducing junction temperature which can beuseful for wavelength stability.

External Light Modulation

An alternative to driving current-based PWM of LEDs is externalmodulation of constant intensity light sources by spatial lightmodulator technology such as Texas Instruments' Digital MicromirrorDevice (DMD) technology trademarked under Digital Light Processor™.Variation in the duty cycle of micromirror deflection of the lightachieves up to 10 bits of grayscale modulation dynamic range. Thesedevices are mass produced in high definition size arrays for videoentertainment products. In such a device, subarrays of mirrors can besynchronously driven to create multiple, independent modulators in onedevice. Smaller array sizes of these devices can be cost effective inlarge quantities. In an example of the lower definition 640×360micromirror array used in pico projectors, subarrays of 90×90 mirrorscan be driven synchronously to create 8 or 9 independent modulatorsexhibiting much larger area than that of a single mirror. This relaxesconstraints on the light collection and focusing optics of the presentinvention. The combination of LEDs with temperature stable centerwavelengths at fixed drive currents and external, low loss modulationmechanisms such as DMDs offers a robust approach to accurate spectralcomposition when using a selection of optimal primary wavelengths andoptimized intensity weights for spectral matching

The comprehensive functionality of a custom display device is shown inFIG. 11 . Included in the user interface is provision for datacommunication such as Bluetooth or USB input. Even keypad entry of codeddata can be an option. Such coded data may be manually transcribed froman email, product catalog entry, or product flyer. If color data isprovided by the vendor as a QR code (encoding over 7,000 digits ofdata), it can be read by a smartphone with appropriate app for export tothe custom display device through Bluetooth, and decoded as an example.Any number of methods of data formatting and input as well known in theprior art can be envisioned for introduction of spectrum informationthrough the user interface. As previously described, the processor 93governs how each device mode (color reproduction or CMF measurement)operates. In the color reproduction mode, the processor 93 provides LEDcontrol electronics 97 with the target LED drive levels and amplitudemixing ratios for energizing the LEDs of the multi-wavelength LED array99 in accordance with the selected mode of operation. The outputs of themulti-wavelength LEDs are spatially homogenized in the color mixingoptics 101. This type of display device can be used to achieve an RGB ora multi-primary implementation. A mechanical selection of function isdepicted with the use of a sliding platform 119. Either the inputoptical axis 113 of the binocular field generator 103 (for colorreproduction display) or the input optical axis 107 of the bipartitefield generator 125 (for CMF measurement) can be selected to receive theoutput 105 of the color mixing optics 101. Such mechanical selection issensed by processor 93 through switch sensor means not shown.

The binocular field generator 103 which creates two equal intensityoptical fields is depicted as a simple combination of beamsplitter 115and folding mirror 117. The left beam 123 and right beam 121 aredirected to the headset display apertures 137 and 139, respectively. Thebipartite field generator 125 depicts a beamsplitter 111 and foldingmirror 109 that create right and left equal intensity bipartite beampaths. The spatial light modulator 112 provides different multiplexingof right and left bipartite beam paths that is synchronized with LEDdrive signals to create the disparate right and left visual fields asobserved by the user in the left monocle or display aperture 137, asrepresented by inset diagram 133. Element 131 represents a mask forlimiting light leakage between right and left bipartite fields.

There is the additional prospect of including in this custom displaydevice, low cost spectrometer functionality that can be used to capturethe consumers lighting environment spectrum. As previously discussed,such spectral information can be combined with the article or productspectrum to produce a total spectrum which, upon display, wouldrepresent how the article or product would be perceived in theconsumer's environment.

This display concept can be extended to the measurement and reproductionof multi-color patterns by sequential measurement of each color andconcurrent display of the reproduced colors within the same visualfield. It would be possible to create a number of smaller instantaneousfields of view within the right and left display apertures and thedifferent colors could be displayed in parallel concurrently or bymultiplexing. A vector of spectral measurements corresponding to the setof colors would be communicated to the consumer

Other Display Primaries Technologies

In addition to the technologies advocated in a preferred embodiment ofthe custom display, use of other technologies is within the scope of thepresently disclosed method and system, among them, laser diodes,narrowband optical filters, and narrowband phosphors.

Low cost, low power laser diodes potentially can be used as primarysources subject to techniques that assure eye safety as employed inlaser-based projectors. Reduced intensities, spoiled spatial and/ortemporal coherence, and divergence angle alteration can be used toachieve this objective.

Narrowband color filters can be used with LEDs to establish stablecenter wavelengths for primaries. If the given LED wavelength varies,the associated filter output center wavelength does not, but outputintensity will vary. Then this intensity variation can be compensated byPWM of the LED.

An emerging technology applicable to the presently disclosed method andsystem comprises narrowband emission phosphors that can have emissionspectra bandwidths of 5 to 10 nm. These phosphors can be pumped withbroadband excitation. The saturation offered by these phosphors cansignificantly increase the color gamut of the displays for the presentcolor reproduction application while assuring center wavelengthstability.

As discussed below, one approach to minimizing observer metamerisminvolves use of a multi-primary display with LED wavelengths determinedby optimization calculations. To determine the relative intensities ofthese LEDs that best match the measured article or product spectrum, themethod of Murakami et al.

Embodiments that do not Require Use of CIE CMFs or Measured Remote UserCMFs

Corresponding to the aforementioned third mode of article or productspectrum measurement, there are two chief categories of embodiments thataltogether avoid the measurement or employment of remote user CMFs. Eachrequires the measurement of the reflectance spectrum (as previouslydiscussed) and subsequent filtering of the consumer's ambient lightingwith a filter mechanism that uses the reflectance spectrum. In the firstsuch embodiment, specific fixed wavelength transmissive filters areoptimally amplitude weighted using the measured reflectance spectrum andemployed to filter the user's ambient light. In a second embodiment,spectrum dispersers are used to spread ambient light across an array ofintensity modulators that weight each discretized wavelength of light bythe associated value of the reflectance spectrum. Thisspectrally-modulated light then is spatially recombined or despread toproduce the article or product color observed with the ambient lighting.

Embodiment Using Fixed Wavelength Filters (Multi Primary Filter)

In this embodiment, the measured reflectance spectrum is approximated bythe same weighted set of “N” primary LED wavelengths as determined bythe method of Murakami et al. described above. However, instead of usingactive sources, i.e. LEDs at these primary wavelengths, passive opticalfilters are used at these wavelengths. Each such filter exhibiting atransmission spectrum that matches the emission spectrum of thecorresponding spectrum LED. These filters, when intensity weighted asdetermined by the aforementioned method of Murakami et al., can be usedto reproduce just the reflectance spectrum of the article or product.Such a composite filter then is used to filter the user's ambient lightand thereby reproduce the color of the article or product in the user'slighting environment. The intensity weighting of the respective filtersis accomplished by external modulation, which effectively can beachieved with the aforementioned DMD technology. FIG. 12 is a functionalblock diagram of this embodiment wherein an illuminator 157 comprising aconstant spectrum light source provides N channels of light with thesame spectrum. Each channel of light is given a pre-computed intensityweighting by a modulator 159 such as a DMD. A user interface 153 permitsthe input of the reflectance spectrum data. A processor 155 uses thisdata to generate the corresponding drive signals to the modulator 159 sothat each channel of light is appropriately intensity modulated.Subsequent to intensity weighting, each channel of light traverses arespective narrow bandpass primary filter of predetermined centerwavelength. These N filters comprise the multiprimary filter 161. Eachchannel of intensity weighted, spectrally-filtered light is spatiallycombined in the color mixing/output optics 163 in order to present theuser with the reproduced article color.

Preferred Embodiment Using Spectrum Dispersers and Spatial LightModulator

In this embodiment of the device, various alternative mechanisms ofspectral dispersion are used in concert with a spatial light modulatorsuch as the DMD discussed above. A digital micromirror device(DMD)-based color reproduction device is one that relies on DMD spatialmodulation of spectrally-dispersed light. The DMD is used to temporallymodulate pixels of light to achieve different average intensities on apixel-by-pixel basis. An example of a commercial DMD is the DigitalLight Processor (DLP®) manufactured by Texas Instruments in various formfactors and pixel densities for use in large screen displays. Uponspectral dispersion of light, DMD pixels can be used to weight therelative intensities of different spectral components of the light.

A DMD-based spectrometer design has been considered for the applicationof hyperspectral imaging in which each pixel of an image can beselectively filtered at high spectral resolution. (S. P. Love et al.,“Full-frame programmable spectral filters based on micromirror arrays,”Journal of Micro/Nanolithography, MEMS, and MOEMS, Vol. 13, No. 1,January-March 2014.). Since pixel (micromirror) sizes in the DMD are onthe order of microns, the DMD is essentially a two-dimensionaldiffraction grating and the implications of the associated diffractioneffects must be taken into account. This is especially true for thefull-frame imaging application addressed by Love et al. The presentapplication is not an imaging one degraded by spectral smearing of animage, but dealing with the existence of DMD diffractive orders remainsan issue.

The diffraction efficiency of the DMD will oscillate as a function ofwavelength (“DMD Optical Efficiency for Visible Wavelengths,” TexasInstruments Application Report, Literature Number: DLPA083A, June2018—Revised May 2019) as diffraction orders containing the most energyvary from the 24-deg micromirror reflection angle as a function ofwavelength. Wavelength dependent spectral distortion caused by thisefficiency variation can be compensated in a calibration of the drivesignal amplitude (micromirror duty cycle) for each pixel (mirror or setof mirrors) of the DMD mapped to a respective wavelength. This willpermit an accurate DMD modulation of the ambient light spectrum by thereflectance spectrum. FIG. 13 is a functional block diagram of thisembodiment in which ambient light is introduced to a spectrum disperser179 by way of input optics 177 which focus and/or collimate the light.The spectrum disperser spatially spreads the different wavelengths oflight across a spatial light modulator 181 which intensity modulateseach wavelength representative region of the spread light by a valuecorresponding to the same wavelength of the reflectance spectrum of anarticle or product. The article or product reflectance spectrum data isinput to this device through data interface 173 which introduces thisspectral information to processor 175. The spectral information isformatted spatially to correspond to the map of wavelengths spreadacross the spatial light modulator 181. The light emanating from spatiallight modulator 181 is spectrally combined in spectral combiner 183 andthe resulting single color light is delivered for human viewing byoutput optics 185.

Variants of a DMD-based color reproduction device are defined bydifferent mechanisms for spectral dispersion, namely, diffraction,refraction, and interference. The following table summarizes theadvantages and disadvantages of these approaches to achieving spectralangle dispersion.

Reflective Virtually Diffraction Imaged Characteristics Prism GratingPhased Array Dispersion Exploits Exploits reflection Exploits Principlevariation from a reflective interference in refractive surface with aamong index with regular grating phase shifted wavelength structurewavefronts Light Broadband Low efficiency Broadband high Efficiency highacross spectrum efficiency efficiency due to multiple diffraction ordersat each wavelength - high efficiency near blaze wavelength WavelengthNonlinear, Large and Extremely large, Dependency highest in theapproximately but constant of Dispersion UV, decreases constantdispersion dispersion from visible to IR Temperature High - large Low-deformation High - large Dependence variation in due to temperaturevariation in of Dispersion refractive refractive index index with withtemperature temperature Higher-Order None Yes - requires Yes - requiresLight higher order higher order light cutout separation filtering StrayLight Low High Low Polarization Low High Low Expense High for Low Lowpolished glass - lesser expense for molded glass and molded polymerDiffraction-Based Variant

Reference is made to FIG. 14 , a schematic diagram of the device 201employing diffractive spectral dispersion. In this design, ambient light203 is focused onto an entrance slit 205 that imparts spatial coherenceto the incoming light. A lens 207 creates a collimated beam of light 209that impinges a first diffraction grating 211. The resulting spectrallydispersed light 213 is filtered by a linear variable filter (LVF) (notshown), well known in the prior art. The LVF serves to remove higherdiffraction orders that would overlap spatially into a singlediffraction order. LVF-filtered beam 213 is spectrally dispersed inangle as it is incident a second diffraction grating 215. Grating 215creates a collimated, spectrally-dispersed beam 217. Each pixel of DMD219 is used to intensity modulate a respective spectral band in beam 217with the corresponding value of a reflectance spectrum at this spectralband. The reflectance spectrum of an example green shoe 221 is depictedas an input to DMD 219 by arrow 223. Spectral data input through a userinterface is converted (by electronics not shown) to the pixelmodulation values loaded into DMD 219. The intensity modulated beam 225reflected from DMD 219 is incident on a third grating 227 which createsa spectrally recombined beam 229. A fourth and final grating 231 forms acollimated beam 233 of uniform color representing the green shoe 221 asseen in the ambient light. Various modalities for display of this lightinclude beam expansion and frosted translucent surfaces. The optics canbe configured to create a one-dimensional spectral spread of the ambientlight or a two-dimensional beam exhibiting columns of light, with eachcolumn corresponding to a different wavelength band.

Subnanometer spectral resolution is achieved by choice of slit size,grating design parameters and the number of micromirrors in the array ofthe DMD employed to modulate the light. The density of micromirrorseffectively quantizes the wavelengths used to represent the reflectancespectrum over the 360 nm-wide visible band. For applications beyondconsumer use requiring higher spectral resolution and or range, a DMDwith larger numbers of micromirrors can be used.

Output optics are used to format this light into a region of singlecolor, spatially uniform light for viewing by the human eye. In thisway, the consumer uses this device to filter an ambient light source(room lighting, outdoor lighting, desktop illumination, solar light,etc.) of their choice by the actual article or product reflectancespectrum so as to view the actual article or product color in highfidelity in the presence of a given ambient light spectrum. Theconsumer's eye and brain will respond as though the actual article orproduct was being viewed in the consumer's chosen ambient light.

Grating spectrometer and grating pulse compressor technologies, wellknown in the prior art, are directly applicable to this embodiment ofthe invention. The tradeoffs between using ruled versus holographicdiffraction gratings are well known in the prior art and can be assessedin a formal reduction to practice of this invention.

FIG. 15 provides a block diagram of a color reproduction device in whichthe spectrum dispersion and spectral combining are achieved by the samefunctional element. Herein, light is introduced though input optics 253to the spectrum disperser/combiner 255 from a given direction. Thislight is spectrally-dispersed and impinges the DMD 259 from this givendirection. It is reflected in the reverse direction and the lightreenters spectrum disperser/combiner 255, but is spectrally-recombinedfor introduction to output optics 257. Data interface 263 and processor261 convey reflectance spectrum information to the DMD 259.

Reference is made to FIG. 16 , a pictorial diagram of a grating-basedimplementation 271 of the block diagram of FIG. 15 . Again, inputambient light 273 is focused through slit 275 and introduced to lens 277to create a collimated beam 279 that traverses a beamsplitter 283.Subsequently, beam 285 is incident on a first diffraction grating 287which creates spectrally dispersed beam 289 (with the aforementioned LVFnot shown). Beam 289 is converted to a collimated beam 293 by a secondgrating 291. DMD 295 intensity modulates each spectral band of beam 293in accordance with the reflectance spectrum data represented by arrow297. It should be pointed out that the DLP™ DMD exhibits mirrorpositions at ±12 degrees for the reflection on and off states,respectively. For simplicity of presentation, FIG. 16 and following donot depict this angular offset from the perpendicular to the DMDenvelope. DMD 295 reflects beam 293 in the reverse direction whereby itundergoes convergence by grating 291 and spectral recombining by grating287. The ambient light, filtered with the reflectance spectrum isprojected along beam 283 for viewing by the user. Given the light lossassociated with use of the beamsplitter 281, consideration is given to amethod to reduce such loss. This is depicted in the schematic diagram ofFIG. 17 which incorporates and optical isolator function. This isachieved by a scheme well known in the prior art that uses a combinationof a polarizing beamsplitter 303 and wideband quarter wave plate 305.The beam propagates in the first direction with one polarization stateand is converted to an orthogonal polarization state for propagation inthe reverse direction. The polarization beamsplitter thereby separatesthe forward and reverse propagating beams. For the two polarizationstates, s and p, the gratings will have different efficiencies, butthere is no polarization-based spectral dispersion so this will notcause spectral distortion.

Refraction-Based Variant

A first refraction-based variant of the device using spectrum dispersionis depicted in the schematic diagram of FIG. 18 . This device 351incorporates prisms in lieu of the gratings in the device of FIG. 14 .An ambient light beam 353 is introduced to a first prism 355 whichcreates a spectrally-dispersed beam 357. Prism 359 collimates beam 357to create beam 361. DMD 363 intensity modulates the spectral bands ofbeam 361 in accordance with the reflectance spectrum data 367representing the color of shoe 365. The reflected, spectrally-dispersedbeam 369 is spatially modulated with an intensity corresponding to eachband of the reflectance spectrum. Prism 371 creates a converging beam371 incident on prism 375 which spatially recombines the spectralcomponents to form beam 373, thereby creating a collimated beam 377 ofuniform color after traversing prism 375.

A second refraction-based variant of the device is shown in FIG. 19 .This device 401 incorporates two prisms in lieu of the two gratings inthe device of FIG. 16 . An ambient light beam 403 traverses abeamsplitter 405 with the transmitted light proceeding along beam path407. Prism 409 creates a spectrally-dispersed beam 411 and prism 413forms collimated beam 415. DMD 417 modulates and retroreflects the beamso that it retraces its path to the beamsplitter 405 whereupon a singlecolor beam 421 is reflected for user viewing.

FIG. 20 depicts the additional use of an optical isolator comprisingpolarization beamsplitter 453 and wideband quarter wave plate 455 toconserve the amount of light useful for color observation.

A compact geometry for a spectral disperser color reproduction deviceborrows from optical pulse compression technology. This implementationusing only one prism is depicted in the pictorial diagram of device 471in FIG. 21 . An ambient light source 473 is focused by lens 475 into aslit 479. The beam 481 exiting slit 479 is reflected from mirror 483along beam path 485 and is incident on prism 487. Beam 489 exiting prism487 is spectrally dispersed and is retrodirected by corner cube mirror491 to form beam 493 that reenters prism 487. Upon exiting prism 487 thebeam 495 is collimated and is reflected from mirror 494 onto DMD 497.Each spectral component of beam 495 is intensity modulated by DMD 497 inaccordance with the reflectance spectrum data 511. Any rejected lightfrom DMD 497 is captured in a light absorbing beam dump 501 at the faceof prism 487. Modulated beam 499 reenters prism 487 and exits asconverging beam 503 which is reflected by corner cube mirror 491 to formbeam 505 which again reenters prism 487 and results in spectrallyrecombined beam 509. Mirror 507 reflects single color beam 509 for userviewing of the reproduced article color seen in ambient light.

Interference-Based Variant

This variant of a spectral dispersion color reproduction device exploitsa Virtually Imaged Phased Array (VIPA). A succinct description of theVIPA comes from Wikipedia: “ . . . the phased array is the opticalanalogue of a phased array antenna at radio frequencies. Unlike adiffraction grating which can be interpreted as a real phased array, ina virtually imaged phased array the phased array is created in a virtualimage. More specifically, the optical phased array is virtually formedwith multiple virtual images of a light source. This is the fundamentaldifference from an Echelle grating, where a similar phased array isformed in the real space. The virtual images of a light source in theVIPA are automatically aligned exactly at a constant interval, which iscritical for optical interference. This is an advantage of the VIPA overan Echelle grating. When the output light is observed, the virtuallyimaged phased array works as if light were emitted from a real phasedarray.”

The VIPA was proposed and named by Shirasaki in 1996. Most recently, inU.S. Pat. No. 10,495,513 to Jean-Ruel et al., a VIPA is disclosed withincreased input light coupling efficiency. This device was developed byLight Machinery, Inc. and commercialized in their series of highspectral resolution spectrometers. The operation of a VIPA can bedescribed with reference to an example application in which the VIPA isused in conjunction with a grating to spatially resolve the stabilizedfrequency comb of a Ti:sapphire femtosecond laser (Diddams et al.,“Molecular fingerprinting with the resolved modes of a femtosecond laserfrequency comb,” Nature, Vol. 445, No. 8, February 2007, pp. 627-630.).FIG. 22A depicts an input beam of light 521 incident to the inputaperture 525 of a VIPA 523. The VIPA is essentially a plane-parallelsolid etalon, where the input beam (focused to a line) is injected at anangle through an uncoated entrance window on the front face. Theremainder of the front face is coated with a high-reflective dielectriccoating, while the back face has a dielectric coating with 96%reflectance. The multiple reflections within the VIPA etalon interferesuch that the exiting beam 527 has its different frequencies emerging atdifferent angles. As with all etalons, the VIPA has a free spectralrange (FSR) determined by its thickness and material index ofrefraction. The output orders are spatially superimposed on each other.This problem is well known in classical spectroscopy, and has beenovercome by using a second dispersive element along an orthogonalspatial dimension. This is depicted in FIG. 22A with grating 529separating the orders into distinct beams 531.

In the example wherein the input comprises radiation from theaforementioned Ti:sapphire femtosecond laser, the output of theVIPA-grating combination consists of a two-dimensional array of thefrequency comb modes as depicted in FIG. 22B, where each ‘dot’represents an individual mode. Within a column (y), which is tilted bythe grating dispersion, the dots are separated by the mode spacing.Within each row (x), the dots are separated by the VIPA free spectralrange.

In the present application, the VIPA-grating combination serves as aspectrometer for generating a two-dimensional distribution of spectrumvalues for ambient light. The (y) axis will exhibit very high resolutionspectral samples within the FSR of the VIPA and each column generated bygrating dispersion will be a separate spectrally adjacent FSR-wideportion of the spectrum. The reflectance spectrum will be mapped to thisgeometry in the DMD for appropriately modulating each wavelength bin ofthe ambient light spectrum. A good guide to the design of VIPAs is foundin the reference to Xiao et al. (Xiao et al., “A Dispersion Law forVirtually Imaged Phased-Array Spectral Dispersers Based on Paraxial WaveTheory,” IEEE Journal of Quantum Electronics, Vol. 40, NO. 4, April2004, pp. 420-426.).

FIG. 23 is a pictorial diagram of a color reproduction device 551 usinga VIPA. Herein an ambient light beam 553 is introduced to beamsplitter555. The transmitted component 557 enters cylindrical lens 559 whichdelivers focused beam 601 to the entrance aperture of VIPA 603. Beam 605exiting the VIPA 603 is spectrally dispersed, but with many overlappingspectral orders. Grating 607 is used to separate these overlappingorders along the cross dimension, resulting in a divergingtwo-dimensional spectral dispersion of light in beam 609. Mirror 611focuses beam 609 onto DMD 615 which modulates the beam using reflectancespectrum information, as previously discussed, and retroreflects themodulated light. As a result, a single color beam 619 emerges frombeamsplitter 555. The optical isolator depicted in previous variants ofthe color reproduction device also can be used with the VIPA-gratingvariant of the device to improve light conservation for the user.

A compact spectral disperser has been designed for endoscopic imaging(Metz et al., “Compact, transmissive two-dimensional spatial disperserdesign with application in simultaneous endoscopic imaging and lasermicrosurgery,” Applied Optics, January 2014, pp. 376-382.) and optimizedfor high spatial resolution, a small device diameter, and a large fieldof view. This design is incorporated in the color reproduction device651 of FIG. 24 . Ambient light 653 is focused on the input aperture ofVIPA 657. The resulting spectrally-dispersed light exhibitingoverlapping orders 659 is introduced to a perpendicularly alignedvolume-holographic grating 663, embedded between two prisms 661 and 662(GRISM). Each FSR portion of the spectrum of this light is fanned out intwo dimensions to form beams 665 which are incident on lens 667 to formcollimated, spectrally dispersed beams 669 that illuminate DMD 671.These collimated beams 669 are intensity modulated by DMD 671 inaccordance with the reflectance spectrum and retrodirected through theseoptics to a beamsplitter (not shown) for observation of the articlecolor in the ambient light.

Reflected Versus Fluorescent Light

Some surfaces exhibit a combination of reflection and fluorescence sothat it appears the reflectance at certain wavelengths is greater thanunity due to downconversion of ultraviolet or even visible light. Sincea fluorescent material absorbs light over a range of wavelengths with acharacteristic excitation range that can reside in the ultraviolet orvisible bands, it is necessary to measure the spectral response of thematerial at each wavelength across the ultraviolet and visible bands.Upon measuring the spectrum of the ambient light that would be used toview an article or product, the complete reflective and fluorescentresponse can be computed for the material. A normalized completespectrum then can be displayed with the DMD-based color reproductiondevice.

Summary of the Methods of the Present Disclosure

The prior description of the devices that support execution of themethods of the present disclosure help to clarify those methods whichare summarized below:

-   -   Use a spectrometer to measure the article total spectrum under        standard illumination and process the measured spectrum to        calculate the drive signals for a multi-primary display using        spectral match optimization under CIE or remote user CMF        constraint. Publish this drive signal information for remote        reproduction on a multi-primary display.    -   Use a spectrometer to measure the article or product total        spectrum and the illumination spectrum and process the spectral        data to produce a reflectance spectrum. Publish the reflectance        spectrum information. Measure the ambient illumination spectrum        in the remote user's environment. Combine this spectrum with the        reflectance spectrum and process the resulting spectrum to        calculate the drive signals for a multi-primary display using        spectral match optimization under CIE or remote user CMF        constraint.    -   Use a spectrometer to measure the article total spectrum under        standard illumination. Publish the total spectrum information.        Measure the remote user's CMFs. Filter the published spectrum        information with the remote user's CMFs. Display resulting        tristimulus values on an RGB display.    -   Use a spectrometer to measure the article total spectrum and the        illumination spectrum and process the spectral data to produce a        reflectance spectrum. Publish the reflectance spectrum        information. Measure the ambient illumination spectrum in the        remote user's environment. Combine this spectrum with the        reflectance spectrum. Measure the remote user's CMFs. Filter the        combined spectrum information with the remote user's CMFs.        Display resulting tristimulus values on an RGB display.    -   Use a spectrometer to measure the article total spectrum under        standard illumination. Publish the total spectrum information.        Measure the remote user's CMFs. Filter the published spectrum        information with the measured remote user's CMFs to produce        tristiumulus values. Calibrate the remote user's display with        the remote user's measured CMFs. Display the color produced by        the tristimulus values on the remote user's CMF-calibrated        display.    -   Use a spectrometer to measure the article total spectrum under        standard illumination. Publish the total spectrum information.        Measure the remote user's CMFs. Filter the published spectrum        information with the measured remote user's CMFs to produce        tristiumulus values. Display the color produced by the        tristimulus values on a custom RGB display.    -   Use a spectrometer to measure the article total spectrum and the        illumination spectrum and process the spectral data to produce a        reflectance spectrum. Publish the reflectance spectrum        information. Measure the ambient illumination spectrum in the        remote user's environment. Combine this spectrum with the        reflectance spectrum. Measure the remote user's CMFs. Filter the        combined spectrum information with the measured remote user's        CMFs to produce tristiumulus values. Display the color produced        by the tristimulus values on the remote user's CMF-calibrated        display.    -   Use a spectrometer to measure the article total spectrum and the        illumination spectrum and process the spectral data to produce a        reflectance spectrum. Publish the reflectance spectrum        information. Measure the ambient illumination spectrum in the        remote user's environment. Combine this spectrum with the        reflectance spectrum. Measure the remote user's CMFs. Filter the        combined spectrum information with the measured remote user's        CMFs to produce tristiumulus values. Display the color produced        by the tristimulus values on a custom RGB display.    -   Use a smartphone camera to display a scene, object, or color        checker pattern and adjust the hues of the smartphone display to        match the hues of the actual scene, object, or color checker        pattern. Use the resulting display calibration data and the        camera spectral responsivity to determine the user's CMFs.        Import the vendor-provided article or product spectrum data and        filter it with the user's CMFs and display the reproduced        article or product color associated with the CMF-filtered        article or product spectrum.    -   Use a spectrometer to measure the article total spectrum and the        illumination spectrum and process the spectral data to produce a        reflectance spectrum. Using the set of primary filters at        wavelengths chosen to minimize metamerism, perform a spectral        match optimization under CIE or remote user CMF constraint to        determine the primary filter weights. Publish these filter        weights for remote reproduction of the reflectance spectrum        implemented with spectral filters. Use the intensity weighted        spectral filters to filter the remote user's ambient lighting to        reproduce article color in a viewer.    -   Use a spectrometer to measure the article total spectrum and the        illumination spectrum and process the spectral data to produce        an article reflectance spectrum. Publish the reflectance        spectrum information. Use a spectrum disperser—digital        micromirror device to filter the remote user's ambient light        with the reflectance spectrum for presentation of article color        in the remote user's ambient light.

Those skilled in the art will understand that a number of variations maybe made in the disclosed embodiments to articulate other variants ofdesigns, all without departing from the scope of the invention, which isdefined solely by the appended claims.

The invention claimed is:
 1. A method of remote reproduction of articleor product color spectrum that permits display to a remote user thearticle or product in true color in avoidance of metamerism, the methodcomprising steps of: a. obtaining spectral information about the articleor product, b. publishing the spectral information for use by the remoteuser, c. reproducing the article or product color spectrum from thepublished spectral information based on a method taken from the groupconsisting of: i. optimizing relative intensities of individualprimaries of a multi-primary display under a constraint that uses thecolor matching functions (CMFs) of CIE standard observers to match thespectral information about the article or product, ii. optimizing therelative intensities of individual primaries of a multi-primary displayunder a constraint that uses the CMFs of the remote user to match thespectral information about the article or product, iii. optimizingrelative weights of individual primaries of a multi-primary spectralfilter under a constraint that uses the CMFs of CIE standard observersto match the spectral information about the article or product andfiltering the remote user's ambient light with the multi primaryspectral filter, iv. optimizing the relative weights of individualprimaries of a multi-primary spectral filter under a constraint thatuses the CMFs of the remote user and filtering the remote user's ambientlight with the multi primary spectral filter, v. optimizing the relativeintensities of individual primaries of a multi-primary display under aconstraint that uses the CMFs of CIE standard observers to match acombination of the spectral information about the article or product andremote user ambient light spectral information, vi. optimizing therelative intensities of individual primaries of a multi-primary displayunder a constraint that uses the CMFs of the remote user to match acombination of the spectral information about the article or product andremote user ambient light spectral information, vii. filtering thespectral information about the article or product with the CMFs of theremote user to create tristimulus values for presentation on an RGBdisplay, viii. filtering a combination of the spectral information aboutthe article or product and remote user ambient light spectralinformation with the CMFs of the remote user to create tristimulusvalues for presentation on an RGB display, ix. displaying on asmartphone camera display that exhibits different hues, a scene, object,or color checker pattern, adjusting the hues of the smartphone displayto match hues of the scene, object, or color checker pattern, usingresulting display calibration data and smartphone camera spectralresponsivity to determine the remote user's CMFs, importing articlespectrum data and filtering it with the remote user's CMFs, anddisplaying a reproduced article color on the smartphone camera display,x. filtering the remote user's ambient light with the spectralinformation about the article or product using a spectrumdisperser-digital micromirror device, and d. displaying colorcorresponding to the reproduced article or product color spectrum on adisplay.
 2. A method of remote reproduction of article or product colorspectrum as recited in claim 1, wherein the spectral information isobtained by spectroscopic means, and published in the form of a)electronic data that can be used in a display device to display thereproduced color, b) printed form that can be manually entered into thedisplay device, c) data published on a website that can beelectronically downloaded for use in a display device or manuallyentered in a display device, or d) emailed data that likewise can beused for display of the product color.
 3. A method of remotereproduction of article or product color spectrum as recited in claim 1,wherein the spectral information of method i. comprises article totalspectrum.
 4. A method of remote reproduction of article or product colorspectrum as recited in claim 1, wherein the spectral information ofmethod ii. comprises article total spectrum.
 5. A method of remotereproduction of article or product color spectrum as recited in claim 1,wherein the spectral information of method iii. comprises article orproduct reflectance spectrum.
 6. A method of remote reproduction ofarticle or product color spectrum as recited in claim 1, wherein thespectral information of method iv. comprises article or productreflectance spectrum.
 7. A method of remote reproduction of article orproduct color spectrum as recited in claim 1, wherein the spectralinformation of method v. comprises article or product reflectancespectrum and the ambient light spectral information comprises an ambientlight spectrum, and the relative intensities of individual primaries ofa multi-primary display are optimized to match results obtained bymultiplying the article or product reflectance spectrum by the ambientlight spectrum.
 8. A method of remote reproduction of article or productcolor spectrum as recited in claim 1, wherein the spectral informationof method vi. comprises article or product reflectance spectrum and theambient light spectral information comprises an ambient light spectrum,and the relative intensities of individual primaries of a multi-primarydisplay are optimized to match results obtained by multiplying thearticle or product reflectance spectrum by the ambient light spectrum.9. A method of remote reproduction of article or product color spectrumas recited in claim 1, wherein the spectral information of method vii.comprises article total spectrum.
 10. A method of remote reproduction ofarticle or product color spectrum as recited in claim 1, wherein thespectral information of method viii. comprises article or productreflectance spectrum and the ambient light spectral informationcomprises an ambient light spectrum, and a combination of the spectralinformation about the article or product and remote user ambient lightspectral information comprising results obtained by multiplying togetherthese two spectra.
 11. A method of remote reproduction of article orproduct color spectrum as recited in method x of claim 1, wherein thespectrum disperser is taken from the group consisting of: a. a set offour prisms, b. a set of four gratings, c. a set of two prisms, d. a setof two gratings, and e. a single prism, the method further comprising:a. focusing ambient light into a spectrum disperser to create spectrallydispersed light, b. directing the spectrally dispersed light to amicromirror device, c. modulating the intensity of thespectrally-dispersed light in a spatial pattern with the micromirrordevice using an article or product reflectance spectrum, therebycreating intensity modulated, spectrally-dispersed light, d. directingthe intensity modulated, spectrally-dispersed light back through thespectrum disperser, to spectrally recombine the light for presentationat a viewing plane.
 12. A method of remote reproduction of article orproduct color spectrum as recited in method x of claim 1, wherein thespectrum disperser comprises a combination of virtual imaged phase array(VIPA) and grating, the method further comprising: a. inputting ambientlight to a beamsplitter thereby splitting the light along a first andsecond path, b. focusing ambient light from the first path into a VIPA,c. dispersing the light spectrally with the VIPA to create a lightspectrally-dispersed along one dimension, thereby creating onedimensionally spectrally-dispersed light, d. directing the onedimensionally spectrally-dispersed light to a grating, e. dispersingspectrally along a second dimension, the one dimensionallyspectrally-dispersed light with the grating to create two-dimensionallyspectrally-dispersed light, f. directing the two dimensionallyspectrally-dispersed light to a focusing optic, g. focusing the twodimensionally spectrally-dispersed light with the focusing optic onto amicromirror device, h. modulating the intensity of the two dimensionallyspectrally-dispersed light in a spatial pattern with the micromirrordevice using an article or product reflectance spectrum, therebycreating an intensity modulated, two dimensionally spectrally-dispersedlight, i. retrodirecting the intensity modulated, two dimensionallyspectrally-dispersed light back through the focusing optic, the grating,the VIPA, and the beamsplitter to a viewing plane.